Method, device, and non-transitory computer-readable recording medium for displaying graphic object on image

ABSTRACT

A method for displaying a graphic object on an image is provided. The method includes the steps of: acquiring, with respect to a first graphic object inputted by a user onto a first image of a person&#39;s posture, attribute information of the first graphic object that defines the first graphic object; deriving, on the basis of a feature point detected from each of the first image and a second image of a person&#39;s posture, a transformation model that defines a transformation relationship between the attribute information of the first graphic object and attribute information of a second graphic object to be displayed on the second image in correspondence to the first graphic object; and displaying, on the basis of the transformation model, the second graphic object at a position on the second image corresponding to a position of the first graphic object on the first image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a national phase of Patent Cooperation Treaty (PCT)International Application No. PCT/KR2021/013321 filed on Sep. 29, 2021,which claims priority to Korean Patent Application No. 10-2020-0138416filed on Oct. 23, 2020. The entire contents of PCT InternationalApplication No. PCT/KR2021/013321 and Korean Patent Application No.10-2020-0138416 are hereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to a method, system, and non-transitorycomputer-readable recording medium for displaying a graphic object on animage.

BACKGROUND

Because learning and practicing proper postures is so important inlearning any sport, people spend a lot of time and money working withcoaches to correct their postures. In order to provide more effectiveeducation in this correction process, a coach often shows a learner animage of the learner's posture and another image of a person's posturethat may be compared to the image (e.g., an image of a professionalathlete taking a posture corresponding to the learner's posture) tocompare and illustrate the two postures.

As an example of related conventional techniques, a technique has beenintroduced which assists a coach to draw a graphic object such as a lineor figure at a specific position on an image of a person's posture tofacilitate comparing and illustrating two postures as described above.However, according to the techniques introduced so far as well as theaforementioned conventional technique, there is a problem that it causesinconvenience to draw a graphic object such as a line or figure at aspecific position on an image of a learner's posture, and then redraw agraphic object corresponding to the above graphic object at a positionon another image of a person's posture corresponding to the specificposition.

SUMMARY OF THE INVENTION

One object of the present invention is to solve all the above-describedproblems in prior art.

Another object of the invention is to acquire, with respect to a firstgraphic object inputted by a user onto a first image of a person'sposture, attribute information of the first graphic object that definesthe first graphic object; derive, on the basis of a feature pointdetected from each of the first image and a second image of a person'sposture, a transformation model that defines a transformationrelationship between the attribute information of the first graphicobject and attribute information of a second graphic object to bedisplayed on the second image in correspondence to the first graphicobject; and display, on the basis of the transformation model, thesecond graphic object at a position on the second image corresponding toa position of the first graphic object on the first image.

Yet another object of the invention is to assist a user to moreconveniently compare and illustrate two postures by, when the userinputs a first graphic object onto a first image of a person's posture,automatically displaying a second graphic object at a position on asecond image corresponding to a position of the first graphic object onthe first image.

The representative configurations of the invention to achieve the aboveobjects are described below.

According to one aspect of the invention, there is provided a methodcomprising the steps of: acquiring, with respect to a first graphicobject inputted by a user onto a first image of a person's posture,attribute information of the first graphic object that defines the firstgraphic object; deriving, on the basis of a feature point detected fromeach of the first image and a second image of a person's posture, atransformation model that defines a transformation relationship betweenthe attribute information of the first graphic object and attributeinformation of a second graphic object to be displayed on the secondimage in correspondence to the first graphic object; and displaying, onthe basis of the transformation model, the second graphic object at aposition on the second image corresponding to a position of the firstgraphic object on the first image.

According to another aspect of the invention, there is provided a systemcomprising: an image management unit configured to acquire, with respectto a first graphic object inputted by a user onto a first image of aperson's posture, attribute information of the first graphic object thatdefines the first graphic object; a transformation model derivation unitconfigured to derive, on the basis of a feature point detected from eachof the first image and a second image of a person's posture, atransformation model that defines a transformation relationship betweenthe attribute information of the first graphic object and attributeinformation of a second graphic object to be displayed on the secondimage in correspondence to the first graphic object; and a graphicobject management unit configured to display, on the basis of thetransformation model, the second graphic object at a position on thesecond image corresponding to a position of the first graphic object onthe first image.

In addition, there are further provided other methods and systems toimplement the invention, as well as non-transitory computer-readablerecording media having stored thereon computer programs for executingthe methods.

According to the invention, it is possible to acquire, with respect to afirst graphic object inputted by a user onto a first image of a person'sposture, attribute information of the first graphic object that definesthe first graphic object; derive, on the basis of a feature pointdetected from each of the first image and a second image of a person'sposture, a transformation model that defines a transformationrelationship between the attribute information of the first graphicobject and attribute information of a second graphic object to bedisplayed on the second image in correspondence to the first graphicobject; and display, on the basis of the transformation model, thesecond graphic object at a position on the second image corresponding toa position of the first graphic object on the first image.

According to the invention, it is possible to assist a user to moreconveniently compare and illustrate two postures by, when the userinputs a first graphic object onto a first image of a person's posture,automatically displaying a second graphic object at a position on asecond image corresponding to a position of the first graphic object onthe first image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically shows the configuration of an entire system fordisplaying a graphic object on an image according to one embodiment ofthe invention.

FIG. 2 specifically shows the internal configuration of a graphic objectdisplay system according to one embodiment of the invention.

FIG. 3A illustratively shows how to display a graphic object on an imageaccording to one embodiment of the invention.

FIG. 3B illustratively shows how to display a graphic object on an imageaccording to one embodiment of the invention.

FIG. 4A illustratively shows how to display a graphic object on an imageaccording to one embodiment of the invention.

FIG. 4B illustratively shows how to display a graphic object on an imageaccording to one embodiment of the invention.

FIG. 5A illustratively shows how general convolution is performedaccording to one embodiment of the invention.

FIG. 5B illustratively shows how depthwise convolution and pointwiseconvolution are performed according to one embodiment of the invention.

DETAILED DESCRIPTION

In the following detailed description of the present invention,references are made to the accompanying drawings that show, by way ofillustration, specific embodiments in which the invention may bepracticed. These embodiments are described in sufficient detail toenable those skilled in the art to practice the invention. It is to beunderstood that the various embodiments of the invention, althoughdifferent from each other, are not necessarily mutually exclusive. Forexample, specific shapes, structures, and characteristics describedherein may be implemented as modified from one embodiment to anotherwithout departing from the spirit and scope of the invention.Furthermore, it shall be understood that the positions or arrangementsof individual elements within each embodiment may also be modifiedwithout departing from the spirit and scope of the invention. Therefore,the following detailed description is not to be taken in a limitingsense, and the scope of the invention is to be taken as encompassing thescope of the appended claims and all equivalents thereof. In thedrawings, like reference numerals refer to the same or similar elementsthroughout the several views.

Hereinafter, various preferred embodiments of the present invention willbe described in detail with reference to the accompanying drawings toenable those skilled in the art to easily implement the invention.

Although the descriptions herein are focused on golf, it will beapparent to those skilled in the art that the present invention may beutilized even for displaying a graphic object on an image of a posturetaken in sports other than golf. For example, the present invention maybe utilized for displaying a graphic object on an image of a baseballswing or a workout or yoga posture.

Although the descriptions herein are focused on detecting joints from animage of a person's posture in order to facilitate understanding, itshould be understood that the present invention is not limited todetecting the joints and may also be utilized for detecting body partsother than the joints.

Configuration of the Entire System

FIG. 1 schematically shows the configuration of the entire system fordisplaying a graphic object on an image according to one embodiment ofthe invention.

As shown in FIG. 1 , the entire system according to one embodiment ofthe invention may comprise a communication network 100, a graphic objectdisplay system 200, and a device 300.

First, the communication network 100 according to one embodiment of theinvention may be implemented regardless of communication modality suchas wired and wireless communications, and may be constructed from avariety of communication networks such as local area networks (LANs),metropolitan area networks (MANs), and wide area networks (WANs).Preferably, the communication network 100 described herein may be theInternet or the World Wide Web (WWW). However, the communication network100 is not necessarily limited thereto, and may at least partiallyinclude known wired/wireless data communication networks, knowntelephone networks, or known wired/wireless television communicationnetworks.

For example, the communication network 100 may be a wireless datacommunication network, at least a part of which may be implemented witha conventional communication scheme such as WiFi communication,WiFi-Direct communication, Long Term Evolution (LTE) communication, 5Gcommunication, Bluetooth communication (including Bluetooth Low Energy(BLE) communication), infrared communication, and ultrasoniccommunication. As another example, the communication network 100 may bean optical communication network, at least a part of which may beimplemented with a conventional communication scheme such as LiFi (LightFidelity).

Next, the graphic object display system 200 according to one embodimentof the invention may function to: acquire, with respect to a firstgraphic object inputted by a user onto a first image of a person'sposture, attribute information of the first graphic object that definesthe first graphic object; derive, on the basis of a feature pointdetected from each of the first image and a second image of a person'sposture, a transformation model that defines a transformationrelationship between the attribute information of the first graphicobject and attribute information of a second graphic object to bedisplayed on the second image in correspondence to the first graphicobject; and display, on the basis of the transformation model, thesecond graphic object at a position on the second image corresponding toa position of the first graphic object on the first image.

The configuration and functions of the graphic object display system 200according to the invention will be discussed in more detail below.

Next, the device 300 according to one embodiment of the invention isdigital equipment capable of connecting to and then communicating withthe graphic object display system 200, and any type of digital equipmenthaving a memory means and a microprocessor for computing capabilities,such as a smart phone, a tablet, a smart watch, a smart band, smartglasses, a desktop computer, a notebook computer, a workstation, apersonal digital assistant (PDAs), a web pad, and a mobile phone, may beadopted as the device 300 according to the invention.

In particular, the device 300 may include an application (not shown) forassisting a user to receive services according to the invention from thegraphic object display system 200. The application may be downloadedfrom the graphic object display system 200 or an external applicationdistribution server (not shown). Meanwhile, the characteristics of theapplication may be generally similar to those of an image managementunit 210, a transformation model derivation unit 220, a graphic objectmanagement unit 230, a communication unit 240, and a control unit 250 ofthe graphic object display system 200 to be described below. Here, atleast a part of the application may be replaced with a hardware deviceor a firmware device that may perform a substantially equal orequivalent function, as necessary.

Configuration of the Graphic Object Display System

Hereinafter, the internal configuration of the graphic object displaysystem 200 crucial for implementing the invention and the functions ofthe respective components thereof will be discussed.

FIG. 2 specifically shows the internal configuration of the graphicobject display system 200 according to one embodiment of the invention.

As shown in FIG. 2 , the graphic object display system 200 according toone embodiment of the invention may comprise an image management unit210, a transformation model derivation unit 220, a graphic objectmanagement unit 230, a communication unit 240, and a control unit 250.According to one embodiment of the invention, at least some of the imagemanagement unit 210, the transformation model derivation unit 220, thegraphic object management unit 230, the communication unit 240, and thecontrol unit 250 may be program modules to communicate with an externalsystem (not shown). The program modules may be included in the graphicobject display system 200 in the form of operating systems, applicationprogram modules, or other program modules, while they may be physicallystored in a variety of commonly known storage devices. Further, theprogram modules may also be stored in a remote storage device that maycommunicate with the graphic object display system 200. Meanwhile, suchprogram modules may include, but are not limited to, routines,subroutines, programs, objects, components, data structures, and thelike for performing specific tasks or executing specific abstract datatypes as will be described below in accordance with the invention.

Meanwhile, the above description is illustrative although the graphicobject display system 200 has been described as above, and it will beapparent to those skilled in the art that at least a part of thecomponents or functions of the graphic object display system 200 may beimplemented in the device 300 or a server (not shown) or included in anexternal system (not shown), as necessary.

First, the image management unit 210 according to one embodiment of theinvention may function to acquire, with respect to a first graphicobject inputted by a user onto a first image of a person's posture,attribute information of the first graphic object that defines the firstgraphic object.

Specifically, according to one embodiment of the invention, attributeinformation of a graphic object refers to information that may definethe graphic object, and may include position information and shapeinformation of the graphic object.

More specifically, according to one embodiment of the invention, theposition information of the graphic object may include information oncoordinates at which the graphic object is positioned on an image of aperson's posture. According to one embodiment of the invention, theinformation on the coordinates of the graphic object may refer toabsolute coordinates on the image (e.g., values of an x-coordinate and ay-coordinate in a two-dimensional image), or refer to relativecoordinates from a specific position on the image (e.g., a position of aright shoulder joint of the person detected on the image).

Further, according to one embodiment of the invention, the shapeinformation of the graphic object may include information on a form,size, length, color, and the like of the graphic object. According toone embodiment of the invention, the form of the graphic object mayinclude a point, a line, an angle represented by two lines, a tetragon,a circle, and the like.

For example, according to one embodiment of the invention, when thegraphic object is rectangular, the attribute information of the graphicobject may include absolute coordinates of a point in the upper leftcorner of the graphic object and absolute coordinates of a point in thelower right corner of the graphic object as the position information ofthe graphic object, and may include the rectangle as the shapeinformation of the graphic object.

As another example, according to one embodiment of the invention, whenthe graphic object is circular, the attribute information of the graphicobject may include absolute coordinates of a center point of the graphicobject and absolute coordinates of one point on the circle as theposition information of the graphic object, and may include the circleand a length of the radius as the shape information of the graphicobject. In this case, when the length of the radius is included in theshape information of the graphic object, the absolute coordinates of theone point on the circle may not be included in the position informationof the graphic object.

Meanwhile, the attribute information of the graphic object according toone embodiment of the invention is not limited to the foregoing, and maybe diversely changed as long as the objects of the invention may beachieved.

Next, the transformation model derivation unit 220 according to oneembodiment of the invention may function to detect a feature point fromeach of the first image of the person's posture and a second image of aperson's posture.

Specifically, the transformation model derivation unit 220 according toone embodiment of the invention may detect a feature point from each ofthe first image and the second image using an artificial neural networkmodel, and the feature point may include at least one of at least onejoint of the person and a golf club. Meanwhile, according to oneembodiment of the invention, the feature point detected from the secondimage may be detected in advance before a feature point is detected fromthe first image, and stored in a server (not shown) or a separatedatabase (not shown).

More specifically, the transformation model derivation unit 220according to one embodiment of the invention may function to deriveprobability information on at least one of a position of at least onejoint of a person and a position of a golf club from an image of theperson's posture using an artificial neural network model, and detect afeature point from the image with reference to the probabilityinformation.

Further, the transformation model derivation unit 220 according to oneembodiment of the invention may generate a probability map (i.e., outputdata of the artificial neural network model) by using the image of theperson's posture as input data of the artificial neural network model.

For example, according to one embodiment of the invention, theprobability map may be a two-dimensional heat map. Further, thetransformation model derivation unit 220 according to one embodiment ofthe invention may generate at least one two-dimensional heat map imagefor each of the at least one joint of the person using the artificialneural network model, and may derive the probability information on thetwo-dimensional position of the at least one joint of the person on thebasis of properties such as the two-dimensional position of the at leastone joint being more likely to correspond to pixels with larger values,among pixels constituting the generated at least one heat map image, orthe position of the at least one joint being less likely to beaccurately specified as pixels with small values are widely distributedin the heat map, and being more likely to be accurately specified aspixels with large values are narrowly distributed in the heat map.

Meanwhile, the above description of deriving the probability informationon the position of the at least one joint of the person may be similarlyapplied to the case where the feature point detected by thetransformation model derivation unit 220 according to one embodiment ofthe invention is a golf club, and thus a detailed description thereofwill be omitted.

Meanwhile, the artificial neural network model according to oneembodiment of the invention may include, for example, a convolutionalneural network (CNN) model, a recurrent neural network (RNN) model, adeep belief network (DBN) model, or an artificial neural network modelin which the foregoing models are combined. However, the artificialneural network model according to one embodiment of the invention is notlimited to those mentioned above, and may be diversely changed as longas the objects of the invention may be achieved.

Further, the artificial neural network model according to one embodimentof the invention may be a model that is light-weighted using depthwiseconvolution and pointwise convolution.

In addition, the artificial neural network model according to oneembodiment of the invention may be a model that is light-weighted usinga light-weighting algorithm such as pruning, weight quantization, andresidual learning.

Specifically, since artificial neural network models commonly used inobject recognition technology require a high level of computingresources to be consumed for a high level of recognition performance, itis often difficult to use such models in environments where only limitedcomputing resources are provided (e.g., mobile devices). Therefore,according to one embodiment of the invention, an artificial neuralnetwork model may be light-weighted using depthwise convolution andpointwise convolution, and the light-weighted artificial neural networkmodel may be used in a mobile device so that at least one joint of aperson may be detected from an image of the person's posture.

Here, the depthwise convolution according to one embodiment of theinvention may refer to a convolution process in which a kernel isapplied for each depth (i.e., each channel) of an input layer, inperforming convolution in the artificial neural network model accordingto one embodiment of the invention. Meanwhile, since the method ofoperation using the applied kernel is the same as that of generalconvolution, a detailed description thereof will be omitted.

Further, the pointwise convolution according to one embodiment of theinvention may refer to a convolution process in which a kernel of size1×1×M (i.e., a kernel of width 1, height 1, and depth M) is applied foreach point of an input layer, in performing convolution in theartificial neural network model according to one embodiment of theinvention.

FIG. 5A illustratively shows how general convolution is performedaccording to one embodiment of the invention.

FIG. 5B illustratively shows how depthwise convolution and pointwiseconvolution are performed according to one embodiment of the invention.

Referring to FIG. 5A, according to one embodiment of the invention, itmay be assumed that the width, height, and depth of an input layer 211are F, F, and N, respectively; the width, height, and depth of eachkernel 212 are K, K, and N, respectively; and the width, height, anddepth of an output layer 213 are F, F, and M, respectively. Here, it isassumed that padding and stride are appropriately sized such that thereis no change in the width and height of the input layer 211 and theoutput layer 213. In this case, in the general convolution, the kernel212 is applied to the input layer 211 to constitute one depth of theoutput layer 213 (through F×F×K×K×N operations), and these operationsare performed for M kernels 212 so that a total of F×F×K×K×N×Moperations are performed.

Referring to FIG. 5B, according to one embodiment of the invention, itmay be assumed that the width, height, and depth of an input layer 221are F, F, and N, respectively; the width, height, and depth of eachkernel 222 in the depthwise convolution are K, K, and 1, respectively;the width, height, and depth of each kernel 224 in the pointwiseconvolution are 1, 1, and N, respectively; and the width, height anddepth of an output layer 225 are F, F, and M, respectively. In thiscase, the kernel 222 is applied for each depth of the input layer 221 toconstitute each depth of an intermediate layer 223 (through F×F×K×K×1×Noperations). Then, the kernel 224 is applied for each point of theintermediate layer 223 to constitute one depth of the output layer 225(through F×F×1×1×N operations), and these operations are performed for Mkernels 224 so that a total of F×F×1×1×N×M operations are performed inthe pointwise convolution. Therefore, according to one embodiment of theinvention, a total of (F×F×K×K×1×N)+(F×F×1×1×N×M) operations areperformed in the depthwise convolution and the pointwise convolution, sothat the amount of operations is reduced compared to the generalconvolution.

Meanwhile, the light-weighting algorithms according to one embodiment ofthe invention are not necessarily limited to the above algorithms (i.e.,the depthwise convolution and the pointwise convolution), and the orderor number of times of applying each of the above algorithms may also bediversely changed.

Meanwhile, the transformation model derivation unit 220 according to oneembodiment of the invention may function to derive, on the basis of thefeature point detected from each of the first image of the person'sposture and the second image of the person's posture, a transformationmodel that defines a transformation relationship between the attributeinformation of the first graphic object inputted onto the first imageand attribute information of a second graphic object to be displayed onthe second image in correspondence to the first graphic object.

Specifically, according to one embodiment of the invention, the secondgraphic object refers to a graphic object corresponding to the firstgraphic object, and may be a graphic object that is, when the firstgraphic object is inputted at a specific position on the first image,automatically displayed at a position on the second image correspondingto the specific position. Further, the second image of the person'sposture corresponds to the first image, and may be an image in which aperson is displayed who takes a posture similar to the posture of theperson displayed in the first image.

FIGS. 3A to 4B illustratively show how to display a graphic object on animage according to one embodiment of the invention.

For example, referring to FIGS. 3A and 3B, it may be assumed that a userinputs a first graphic object 310 a at a specific position on a firstimage (FIG. 3A). In this case, the specific position may be specified onthe basis of attribute information of the first graphic element 310 a(e.g., absolute or relative coordinates of a point 311 a, absolute orrelative coordinates of a point 312 a, and a rectangle). Further, asecond graphic object 310 b corresponding to the first graphic object310 a may be automatically displayed at a position on a second image(FIG. 3B) corresponding to the specific position. This function may beperformed on the basis of a transformation model according to oneembodiment of the invention, and deriving the transformation model willbe described in detail later.

Meanwhile, when the user inputs a graphic object 320 a or 330 a at aspecific position on the first image (FIG. 3A), a graphic object 320 bor 330 b may be displayed on the second image (FIG. 3B) in a mannersimilar to the above description, and thus a detailed descriptionthereof will be omitted.

As another example, referring to FIGS. 4A and 4B, it may be assumed thata user inputs a first graphic object 410 a at a specific position on afirst image (FIG. 4A). In this case, the specific position may bespecified on the basis of attribute information of the first graphicelement 410 a (e.g., absolute or relative coordinates of a point 411 a,absolute or relative coordinates of a point 412 a, and a straight line).Further, a second graphic object 410 b corresponding to the firstgraphic object 410 a may be automatically displayed at a position on asecond image (FIG. 4B) corresponding to the specific position.

Meanwhile, when the user inputs a graphic object 420 a or 430 a at aspecific position on the first image (FIG. 4A), a graphic object 420 bor 430 b may be displayed on the second image (FIG. 4B) in a mannersimilar to the above description, and thus a detailed descriptionthereof will be omitted.

Meanwhile, the transformation model derived by the transformation modelderivation unit 220 according to one embodiment of the invention definesa transformation relationship between the attribute information of thefirst graphic object and the attribute information of the second graphicobject, and the transformation relationship may be defined on the basisof the feature point detected from each of the first image and thesecond image.

Specifically, when the feature point detected from each of the firstimage and the second image by the transformation model derivation unit220 according to one embodiment of the invention is at least one jointof a person, the transformation model derivation unit 220 according toone embodiment of the invention may make at least one joint detectedfrom the first image correspond to at least one joint detected from thesecond image (e.g., make first to n^(th) joints detected from the firstimage correspond to first to n^(th) joints detected from the secondimage, respectively), and derive a transformation model by defining arelationship between the corresponding joints with reference toinformation on absolute or relative coordinates of the correspondingjoints. That is, according to one embodiment of the invention, thetransformation model may refer to a function that transforms a featurepoint detected from the first image into a feature point correspondingto the feature point detected from the first image, among feature pointsdetected from the second image.

For example, the transformation model derivation unit 220 according toone embodiment of the invention may derive a least squares method-basedaffine transformation matrix as the transformation model, with referenceto the information on absolute or relative coordinates of thecorresponding joints.

However, the transformation model derived by the transformation modelderivation unit 220 according to one embodiment of the invention is notlimited to the least squares method-based affine transformation matrixas described above (e.g., a method other than the least squares methodmay be employed or a linear or non-linear function other than the matrixmay be derived as the transformation model, or the transformation modelmay be derived on the basis of an image warping technique), and may bediversely changed as long as the objects of the invention may beachieved.

Meanwhile, the transformation model derivation unit 220 according to oneembodiment of the invention may function to derive a transformationmodel for each group of feature points that are positionally associatedamong the feature points.

Specifically, the feature points detected by the transformation modelderivation unit 220 according to one embodiment of the invention may begrouped into feature points that are positionally adjacent, and atransformation model may be derived for each of the grouped featurepoints.

For example, when the feature points detected by the transformationmodel derivation unit 220 according to one embodiment of the inventionare joints and a golf club, the transformation model derivation unit 220according to one embodiment of the invention may group the featurepoints by the joints detected from the head region, joints detected fromthe upper body region (including arms), joints detected from the lowerbody region, and golf club, and derive a transformation model for eachof the grouped feature points. That is, according to one embodiment ofthe invention, it is possible to derive a single transformation modelfor all joints of a person, but it is also possible to derivetransformation models for joints of the person's head region, joints ofthe person's upper body region, joints of the person's lower bodyregion, and a golf club, respectively.

Meanwhile, when the user selects a frame of a specific posture of aperson from a first image of the person's posture, the image managementunit 210 according to one embodiment of the invention may function tomodify a second image of a person's posture such that the frame of thespecific posture is displayed in the second image before the user inputsa first graphic object onto the first image.

Specifically, according to one embodiment of the invention, the user mayselect a frame of a specific posture of a person from a first image ofthe person's posture, and the image management unit 210 according to oneembodiment of the invention may identify to which posture the specificposture corresponds on the basis of a feature point detected from thefirst image by the transformation model derivation unit 220 according toone embodiment of the invention. Further, the image management unit 210according to one embodiment of the invention may extract a framecorresponding to the identified posture from a second image of aperson's posture on the basis of a feature point detected from thesecond image by the transformation model derivation unit 220 accordingto one embodiment of the invention, and modify the second image suchthat the frame is displayed in the second image.

For example, according to one embodiment of the invention, it may beassumed that the first image and the second image are videos of persons'golf swing postures. Further, in general, a golf swing may be composedof eight stages of partial motions such as an address, a takeaway, aback swing, a top-of-swing, a down swing, an impact, a follow-through,and a finish.

Continuing with the example, referring to FIGS. 4A and 4B, when the userselects a frame of a person's address posture (FIG. 4A) from the firstimage, the image management unit 210 according to one embodiment of theinvention may extract a frame of a person's address posture (FIG. 4B)from the second image, and modify the second image such that theextracted frame of the address posture is automatically displayed in thesecond image.

Meanwhile, the golf swing according to one embodiment of the inventionis not necessarily separated into the eight stages as described above.That is, it may be separated to further include detailed stagesconstituting each of the eight stages, or such that at least some of theeight stages constitute one stage.

Next, the graphic object management unit 230 according to one embodimentof the invention may function to display, on the basis of thetransformation model derived by the transformation model derivation unit220 according to one embodiment of the invention, the second graphicobject at a position on the second image of the person's posturecorresponding to a position of the first graphic object on the firstimage of the person's posture.

For example, referring to FIGS. 3A and 3B, by applying thetransformation model derived by the transformation model derivation unit220 according to one embodiment of the invention to (e.g., multiplyingthe affine transformation matrix by) position information of the firstgraphic object 310 a inputted onto the first image (FIG. 3A) (e.g.,absolute or relative coordinates of the point 311 a and absolute orrelative coordinates of the point 312 a), position information of thesecond graphic object 310 b corresponding to the first graphic object310 a (e.g., absolute or relative coordinates of a point 311 b andabsolute or relative coordinates of a point 312 b) may be calculated.Further, the graphic object management unit 230 according to oneembodiment of the invention may function to display the second graphicobject 310 b at a position on the second image (FIG. 3B) correspondingto a position of the first graphic object 310 a on the first image (FIG.3A), on the basis of the calculated position information of the secondgraphic object 310 b and shape information of the second graphic object310 b (i.e., a rectangle)

Meanwhile, when the first graphic object 320 a or 330 a is inputted ontothe first image (FIG. 3A), the graphic object management unit 230according to one embodiment of the invention may also display the secondgraphic object 320 b or 330 b on the second image in the same manner asabove, and thus a detailed description thereof will be omitted.

Meanwhile, when the transformation model derivation unit 220 accordingto one embodiment of the invention derives a transformation model foreach group of feature points that are positionally associated among thefeature points respectively derived from the first image and the secondimage, the graphic object management unit 230 according to oneembodiment of the invention may function to display the second graphicobject on the basis of the transformation model derived for a group offeature points positionally associated with the first graphic object.

According to one embodiment of the invention, by deriving transformationmodels for respective groups of positionally associated feature points,and displaying the second graphic object on the basis of thetransformation model derived for a group of feature points positionallyassociated with the first graphic object among the derivedtransformation models, the second graphic object may be displayed at amore accurate position than when a single transformation model isderived for all the feature points.

For example, when the feature points derived by the transformation modelderivation unit 220 according to one embodiment of the invention are aperson's joints, a transformation model for joints of the person's headregion, a transformation model for joints of the person's upper bodyregion, and a transformation model for joints of the person's lower bodyregion may be respectively derived. Further, referring to FIGS. 3A and3B, the graphic object management unit 230 according to one embodimentof the invention may display the second graphic object 310 b on thebasis of a transformation model derived for a group of feature pointspositionally associated with the first graphic object 310 a, i.e., thetransformation model for the joints of the person's head region.

As another example, referring to FIGS. 3A and 3B, the graphic objectmanagement unit 230 according to one embodiment of the invention maydisplay the second graphic object 320 b or 330 b on the basis of atransformation model derived for a group of feature points positionallyassociated with the first graphic object 320 a or 330 a, i.e., thetransformation model for the joints of the person's lower body region.

As yet another example, referring to FIGS. 4A and 4B, the graphic objectmanagement unit 230 according to one embodiment of the invention maydisplay the second graphic object 410 b or 420 b on the basis of atransformation model derived for a group of feature points positionallyassociated with the first graphic object 410 a or 420 a, i.e., thetransformation model for the joints of the person's upper body region.

As still another example, referring to FIGS. 4A and 4B, the graphicobject management unit 230 according to one embodiment of the inventionmay display the second graphic object 430 b on the basis of atransformation model derived for a group of feature points positionallyassociated with the first graphic object 430 a, i.e., a transformationmodel for the golf club.

Next, the communication unit 240 according to one embodiment of theinvention may function to enable data transmission/reception from/to theimage management unit 210, the transformation model derivation unit 220,and the graphic object management unit 230.

Lastly, the control unit 250 according to one embodiment of theinvention may function to control data flow among the image managementunit 210, the transformation model derivation unit 220, the graphicobject management unit 230, and the communication unit 240. That is, thecontrol unit 250 according to the invention may control data flowinto/out of the graphic object display system 200 or data flow among therespective components of the graphic object display system 200, suchthat the image management unit 210, the transformation model derivationunit 220, the graphic object management unit 230, and the communicationunit 240 may carry out their particular functions, respectively.

The embodiments according to the invention as described above may beimplemented in the form of program instructions that can be executed byvarious computer components, and may be stored on a computer-readablerecording medium. The computer-readable recording medium may includeprogram instructions, data files, and data structures, separately or incombination. The program instructions stored on the computer-readablerecording medium may be specially designed and configured for thepresent invention, or may also be known and available to those skilledin the computer software field. Examples of the computer-readablerecording medium include the following: magnetic media such as harddisks, floppy disks and magnetic tapes; optical media such as compactdisk-read only memory (CD-ROM) and digital versatile disks (DVDs);magneto-optical media such as floptical disks; and hardware devices suchas read-only memory (ROM), random access memory (RAM) and flash memory,which are specially configured to store and execute programinstructions. Examples of the program instructions include not onlymachine language codes created by a compiler, but also high-levellanguage codes that can be executed by a computer using an interpreter.The above hardware devices may be changed to one or more softwaremodules to perform the processes of the present invention, and viceversa.

Although the present invention has been described above in terms ofspecific items such as detailed elements as well as the limitedembodiments and the drawings, they are only provided to help moregeneral understanding of the invention, and the present invention is notlimited to the above embodiments. It will be appreciated by thoseskilled in the art to which the present invention pertains that variousmodifications and changes may be made from the above description.

Therefore, the spirit of the present invention shall not be limited tothe above-described embodiments, and the entire scope of the appendedclaims and their equivalents will fall within the scope and spirit ofthe invention.

What is claimed is:
 1. A method for displaying a graphic object on animage, the method comprising the steps of: acquiring, with respect to afirst graphic object inputted by a user onto a first image of a person'sposture, attribute information of the first graphic object that definesthe first graphic object; deriving, on the basis of a feature pointdetected from each of the first image and a second image of a person'sposture, a transformation model that defines a transformationrelationship between the attribute information of the first graphicobject and attribute information of a second graphic object to bedisplayed on the second image in correspondence to the first graphicobject; and displaying, on the basis of the transformation model, thesecond graphic object at a position on the second image corresponding toa position of the first graphic object on the first image.
 2. The methodof claim 1, wherein in the acquiring step, when the user selects a frameof a specific posture of the person from the first image, the secondimage is modified such that the frame of the specific posture isdisplayed in the second image before the first graphic object isinputted.
 3. The method of claim 1, wherein the feature point is atleast one of at least one joint of the person and a golf club detectedfrom each of the first image and the second image using an artificialneural network model.
 4. The method of claim 1, wherein in the derivingstep, the transformation model is derived for each group of featurepoints that are positionally associated among the feature points, andwherein in the displaying step, the second graphic object is displayedon the basis of the transformation model derived for a group of featurepoints positionally associated with the first graphic object.
 5. Anon-transitory computer-readable recording medium having stored thereona computer program for executing the method of claim
 1. 6. A system fordisplaying a graphic object on an image, the system comprising: an imagemanagement unit configured to acquire, with respect to a first graphicobject inputted by a user onto a first image of a person's posture,attribute information of the first graphic object that defines the firstgraphic object; a transformation model derivation unit configured toderive, on the basis of a feature point detected from each of the firstimage and a second image of a person's posture, a transformation modelthat defines a transformation relationship between the attributeinformation of the first graphic object and attribute information of asecond graphic object to be displayed on the second image incorrespondence to the first graphic object; and a graphic objectmanagement unit configured to display, on the basis of thetransformation model, the second graphic object at a position on thesecond image corresponding to a position of the first graphic object onthe first image.
 7. The system of claim 6, wherein the image managementunit is configured to, when the user selects a frame of a specificposture of the person from the first image, modify the second image suchthat the frame of the specific posture is displayed in the second imagebefore the first graphic object is inputted.
 8. The system of claim 6,wherein the feature point is at least one of at least one joint of theperson and a golf club detected from each of the first image and thesecond image using an artificial neural network model.
 9. The system ofclaim 6, wherein the transformation model derivation unit is configuredto derive the transformation model for each group of feature points thatare positionally associated among the feature points, and wherein thegraphic object management unit is configured to display the secondgraphic object on the basis of the transformation model derived for agroup of feature points positionally associated with the first graphicobject.